1. Field of the Invention
The present invention relates to pointing devices for cursors on video display screens in a data processing environment. More particularly, the present invention relates to an optical system, device, and method for imaging a surface to perceive a displacement of the surface without having mechanically moving parts or without requiring a specially patterned surface.
2. Description of the Related Art
Pointing devices, such as a mouse or a trackball, are well known peripheral devices in data processing environments. Pointing devices allow for cursor manipulation on a visual display screen of a personal computer or workstation, for example. Cursor manipulation includes actions such as rapid relocation of a cursor from one area of the display screen to another area or selecting an object on a display screen.
In a conventional electromechanical mouse environment, a user controls the cursor by moving the electromechanical mouse over a reference surface, such as a rubber mouse pad so that the cursor moves on the display screen in a direction and a distance that is proportional to the movement of the electromechanical mouse. Typically, the conventional electromechanical mouse consisted of a mechanical approach where a ball is primarily located within the mouse housing and a portion of the ball is exposed to come in contact with the reference surface so that the ball may be rotated internally within the housing.
The ball of the conventional electromechanical mouse contacts a pair of shaft encoders. The rotation of the ball rotates the shaft encoders, which include an encoding wheel that has multiple slits. A light emitting diode (“LED”), or similar light source, is positioned on one side of the encoding wheel, while a phototransistor, or similar photosensor, is positioned opposite to the LED. When the ball rotates, the rotation of the encoding wheel results in a series of light pulses, from the LED shining through the slits, that are detected by the phototransistor. Thus, the rotation of the ball is converted to a digital representation which is then used to move the cursor on the display screen.
The conventional electromechanical mouse is a relatively accurate device for cursor manipulation. The electromechanical mouse, however, has drawbacks associated with many other devices that have mechanical parts. Namely, over time the mechanical components wear out, become dirty, or simply break down so that the cursor can no longer be accurately manipulated, if at all.
An optical mouse reduces, and in some instances eliminates, the number of mechanical parts. A conventional optical mouse uses a lens to generate an image of a geometric pattern located on an optical reference pad. The conventional optical mouse uses a light beam to illuminate an optical reference pad having a special printed mirror geometric pattern. The geometric pattern is typically a grid of lines or dots that are illuminated by the light source and then focused by a lens on a light detector in the conventional optical mouse.
Typically, the grids are made up of orthogonal lines with vertical and horizontal lines that are printed in different colors and so that when the grid is illuminated, the grid reflects light at different frequencies. The colors absorb light at different frequencies so that optical detectors of the optical mouse can differentiate between horizontal and vertical movement of the conventional optical mouse. The photodetector picks up a series of light-dark impulses that consist of reflections from the printed mirror surface and the grid lines and converts the impulses into square waves. A second LED and photodetector, mounted orthogonally to the first, is used to detect motion in an orthogonal direction. The conventional optical mouse counts the number of impulses created by its motion and converts the result into motion information for the cursor.
The conventional optical mouse provides the advantage of reducing or eliminating the number of mechanical parts. The conventional optical mouse, however, has several drawbacks. One problem with the conventional optical mouse is that it requires an optical pad as described above. To eliminate the optical pad, a coherent light source was used with the conventional optical mouse. The coherent light source illuminated the surface directly below the mouse on most surfaces, except mirror-like surfaces. The use of a coherent light source, however, produced more problems.
The first problem the conventional coherent light optical mouse incurs is from the use of coherent light and speckles. Speckles are a phenomenon in which light from a coherent source is scattered by a patterned surface, such as the grid, to generate a random-intensity distribution of light that gives the surface a granular appearance. In the conventional coherent light optical mouse it is necessary to generate images of speckles to replace the optical pad. The imaging resolution is given by a photosensor pitch, e.g., the distance between two neighboring pixels or the periodicity, Λ, of the detector, a value that typically ranges from 10 micrometers to 500 micrometers. Elements in the image plane having a size smaller than this periodicity are not properly detected.
A pattern is improperly detected by an imaging device when it is too small. The image is ambiguous if the pattern is smaller than the pixel size. A measure of speckle size, or more precisely speckle average diameter Δ, can be shown as Δ=(2/π) (λ/AP), where λ is the light wavelength and AP is a measure of an aperture of the optical system. The aperture of the optical system may be defined as AP=(wp/di), where wp is half the diameter of the aperture and di is the distance from the lens to the image plane.
Conventional coherent light optical systems found in the conventional coherent light optical mouse devices exhibit AP values in the range of 0.2 to 0.8. The maximal speckle size is then approximately 10λ. For commercially available coherent light sources (λ=0.6 to 0.96 micrometers), imaging such a small pattern is currently not achievable at full resolution with current semiconductor technology. Thus, ambiguous and hard to interpret data is read from the sensor when a speckle is smaller than the imaging resolution. This, in turn, leads to erroneous displacement estimates that adversely affect system performance by producing an erroneous displacement sign value.
Conventional optical systems that use a coherent light source produce an illumination spot that must be correctly aligned with a sensor to generate a speckled surface image. Mechanical positioning of the illumination spot is achieved with some tolerance, such that the illuminated spot image on the image plane must be wider than the sensor to make sure the sensor is fully covered by the image of the illumination spot. Having a wide spot results in a reflected spot having a reduced power intensity that the photosensor array must detect. Thus, attempts by conventional optical systems to solve position tolerance, i.e., misalignment, were accompanied by a loss of reflected light that can be captured by the photosensor array.
Another problem with conventional optical pointing devices based on speckle image analysis is sensitivity of an estimation scheme to statistical fluctuations. Because speckles are generated through phase randomization of scattered coherent light, the speckle pattern has a defined size on average, but can exhibit local patterns not consistent with its average shape. Therefore, it is unavoidable for the system to be locally subjected to ambiguous or hard to interpret data, such as where a speckle count observed by the imaging system is small.
An additional problem in conventional optical pointing devices is attaining a small displacement resolution without significantly increasing costs due to increased hardware complexities and increased computational loads. Various methods exist to estimate relative displacement from the analysis of two images of a moving target based on correlation techniques. Typically the correlation between the newly acquired image and the previous image is computed, and the estimated displacement between the two images is found at the spatial coordinates where a peak of the correlation function occurs. An exhaustive search of the peak value is possible after all values of the cross-correlation function are computed.
New images are acquired on a regular basis, at an acquisition rate allowing at least one common part of the image to be included in two successive snapshots, even at high speed. The smallest resolvable displacement, or displacement resolution, is the image resolution, e.g., the photodetector array periodicity Λ, divided by the optical magnification, mag, where mag=(di/do), and di, do are defined as the image distance and the object distance, respectively, as referenced to the lens position.
For even higher displacement resolutions, sub-pixel displacement can be obtained through interpolation by a factor I, however with an excessive increase of computations. Evaluations of the cross-correlation function of two images of size M×M requires roughly 4(M4) Multiply-And-Accumulate (MACs), which translates into 4 (M4)T_acq instructions-per-second (MIPs/1,000,000), where T_acq is the time period between two acquisitions. Typically, T_acq is between 50 microseconds and 1 millisecond. Such large computational load required costly and power hungry digital hardware which is difficult to integrate in a small hand held pointing device.
One more problem with conventional optical pointing devices based on cross-correlation detection is that they are insensitive to displacement occurring when the pointing device speed is lower than the image resolution divided by (mag*T_acq), that is for a displacement smaller than a pixel. Any diagonal displacement at low speed may be registered along one direction and ignored along the other depending on the two displacement components compared to the detection limit. This effect translates into the cursor being “snapped” along the fastest moving direction.
Therefore, there is a need for a system and method that (1) provides for detection of motion of an optical pointing device relative to a surface; (2) provides an optical detection system with an optical sensing assembly having an optical element with an artificially limited aperture that is matched with a photosensor array to generate a speckle image and an image data signal therefrom; (3) provides an optical detection system with an optical sensing assembly having one or more lenses optically matched with one or more photosensor arrays to generate a speckle image and an image data signal therefrom; and (4) provides a method for generating an unambiguous image data signal to determine displacement relative to a surface.